arXiv:1508.03590v2 [cs.GR] 7 Dec 2015 


Light-Field Microscopy with a Consumer Light-Field Camera 


Lois Mignard-Debise 
INRIA, LP2N 
Bordeaux, France 

http://manao.inria.fr/perso/~Imignard/ 


Ivo Ihrke 
INRIA, LP2N 
Bordeaux, France 


Abstract 

We explore the use of inexpensive consumer light- 
field camera technology for the purpose of light-field mi¬ 
croscopy. Our experiments are based on the Lytro (first gen¬ 
eration) camera. Unfortunately, the optical systems of the 
Lytro and those of microscopes are not compatible, lead¬ 
ing to a loss of light-field information due to angular and 
spatial vignetting when directly recording microscopic pic¬ 
tures. We therefore consider an adaptation of the Lytro op¬ 
tical system. 

We demonstrate that using the Lytro directly as an oc¬ 
ular replacement, leads to unacceptable spatial vignetting. 
However, we also found a setting that allows the use of the 
Lytro camera in a virtual imaging mode which prevents the 
information loss to a large extent. We analyze the new vir¬ 
tual imaging mode and use it in two different setups for im¬ 
plementing light-field microscopy using a Lytro camera. As 
a practical result, we show that the camera can be used for 
low magnification work, as e.g. common in quality control, 
surface characterization, etc. We achieve a maximum spa¬ 
tial resolution of about 6.25/im, albeit at a limited SNR for 
the side views. 

1. Introduction 

Light-field imaging is a new tool in the field of digital 
photography. The increasing interest is shown by the recent 
development of several hardware systems on the consumer 
market (Lytro, Raytrix, Picam), and applications in the re¬ 
search domain (stereo-vision, panoramic imaging, refocus¬ 
ing). The commercial systems are reliable, functional and 
inexpensive. However, they are designed for the imaging 
of macroscopic objects. In this article, we explore the use 
of commercial light-field cameras for microscopic imaging 
applications. 

The light-field microscope has been introduced and im¬ 
proved by Levoy et al. 19, 10, 2]. While its conceptual de¬ 
tails are well understood, its practical implementation re¬ 
lies on the fabrication of a custom micro-lens array, which 


presents a hurdle for experimenting with the technology. In 
this article, we demonstrate the use of the Lytro camera, 
an inexpensive consumer-grade light-field sensor, for mi¬ 
croscopic work. We achieve an inexpensive and accessible 
means of exploring light-field microscopy with good qual¬ 
ity, albeit at a reduced optical magnification. 

As we show, the major problem in combining the Lytro 
and additional magnification optics (in addition to f-number 
matching), is the loss of information due to spatial vi¬ 
gnetting. Our main finding is the possibility of using the 
Lytro in what we term an inverse regime: in this setting the 
camera picks up a virtual object that is located far behind its 
imaging optics. To our knowledge, this is the first time that 
such a light-field imaging mode is described. 

We investigate two different setups based on this inverse 
regime that do not suffer from spatial vignetting: 1) Our 
first option enables the use of the Lytro camera in conjunc¬ 
tion with an unmodified microscope by designing an optical 
matching system. 2) The second option uses the Lytro be¬ 
hind a standard SLR lens in a macrography configuration to 
achieve macro light-field photography. 

The paper is organized as follows. In Section 3, we study 
the compatibility of both optical systems involved in the 
imaging process: the light-field camera and the microscope. 
We discuss the implications of the combination of both sys¬ 
tems in terms of both spatial and angular resolution. We 
then explore the Lytro main optical system and show how to 
adapt it to the microscopic imaging context. Sects. 5 and 6. 
Finally, we evaluate and compare the different solutions and 
present application scenarios. Sect. 7. 

2. Related Work 

Light-field imaging requires the acquisition of a large 
number of viewpoints of a single scene. Two types of ap¬ 
proaches exist, either using multiple sensors or a single 
sensor in conjunction with temporal or spatial multiplexing 
schemes. 

Taking a picture with a conventional camera is similar to 
a 2D slicing of the 4D light-field. Repeating this operation 
with a planar array of cameras offers sufficient data to esti- 
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Figure 1. Left: Microscope setup with the Lytro camera on top of two additional SLR lenses. Images of several samples have been taken 
under different illumination conditions. Magnification is 3.0. Top and bottom left: brushed steel from scissors blade with lighting from the 
right and bottom. Top middle left: Scratched surface of a piece of metal. Bottom middle left: Plastic surface with highly retro-reflective 
properties. The material is made of micro bubbles of transparent plastic that are invisible to the naked eye. Top middle right: Fabric with 
a hexagonal structure. The lighting is coming from the side and casts shadows and strong highlights on the three-dimensional structure 
of the fabric. Bottom middle right: Highly retro-reflective material from security reflective tape. Top right: Tonsil tissue with bright field 
illumination. Bottom right: Pins of a electronic component on a circuit board. 


mate the light-field function. Calibration, synchronization, 
as well as the available bandwidth of the camera hardware 
are the major determining features of this approach. More 
details can be found, e.g., in [25, 23]. The strongest lim¬ 
itation of this approach for microscopic applications is the 
large size of the corresponding setups. 

An alternative method for light-field capture is to multi¬ 
plex the different views onto a single sensor. 

Temporal multiplexing is based on taking several pictures 
over time after moving the camera around static scenes. The 
movements can be a translation or rotation of the camera. 
Alternatively, mirrors [5, 19] can be moved to generate ad¬ 
ditional virtual viewpoints. Another alternative implemen¬ 
tation are dynamic apertures [11]. 

Spatial multiplexing allows to record dynamic scenes. 
Parallax barriers and integral imaging [12] are historically 
the first approaches to spatially multiplex the acquisition 
of a light-field, trading spatial resolution for angular reso¬ 
lution. A modern elaboration of this approach where the 
sensor and a micro-lens array are combined to form an in¬ 
camera light-field imaging system is the Hand-Held Plenop- 
tic Camera [17]. Alternatively, sensor masks [22, 21] or a 
light pipe [15] can be arranged such that in-camera light- 
fields can be recorded. Other methods use external arrays 
of mirrors instead of lenses [7], or external lens arrays [4]. 

Microscopy is a vast subject and many different illumi¬ 
nation and observation schemes have been developed in the 


past. A general overview is given in [16]; a comprehen¬ 
sive review of microscopy techniques, including light-field 
microscopy, for the neuro-sciences can be found in [24]. 
Light-field microscopy was introduced by Levoy et al. [9] 
and later augmented with light-field illumination [10]. Re¬ 
cently, addressing the large spatial resolution loss implicit 
in LFM, the group has shown that computational super¬ 
resolution can be achieved outside the focal plane of the 
microscope [2, 3]. Another super-resolution scheme is com¬ 
bining a Shack-Hartmann wavefront sensor and a standard 
2D image to compute a high-resolution microscopic light- 
field [14]. LFM has been applied to polarization studies of 
mineral samples [18] and initial studies for extracting depth 
maps from the light-field data have been performed in mi¬ 
croscopic contexts [8, 20]. Most of the work today uses the 
same optical configuration that was introduced in the origi¬ 
nal implementation [9] . With this article, we aim at provid¬ 
ing an inexpensive means of experimenting with LFM. 

3. Background & Problem Statement 
3.1. Light-field Microscopy 

The main function of an optical microscope is to mag¬ 
nify small objects so that they can be observed with the 
naked eye or a camera sensor. Light-field capabilities such 
as changing the viewpoint, focusing after taking the pic¬ 
ture, and achieving 3D reconstruction of microscopic sam- 





















pies rely on the number of view points that can be measured 
from a scene. This number is directly linked to the object- 
side numerical aperture NAq of the imaging system that is 
used as an image-forming system in front of the micro-lens 
array of the light-field sensor and the micro-lens f-number. 
The object-side numerical aperture is defined as 

NAo = nsin{a)^ (1) 


where n is the index of the material in object space (usu¬ 
ally air, i.e. n = 1). The numerical aperture quantifies the 
extent of the cone of rays originating at an object point and 
being permitted into the optical system (see Fig. 2). Mi¬ 
croscope objectives usually have a high NAq, because it is 
directly linked to better optical resolution and a shallower 
depth of field. A high NAq is also important for light-field 
microscopy as the base-line of the light-field views is di¬ 
rectly linked to it. 

Details on how to design a light-field microscope can be 
found in [9] . The most important aspect is that the f-number 
of the micro-lens array matches the f-number of the micro¬ 
scope objective. The f-number F of any optical system is 
defined as the ratio of its focal length / over the diameter D 
of its entrance pupil (see Fig. 2) 


For a microscope of magnification M^icroscope and numer¬ 
ical aperture NAq, a more appropriate equation taking the 
finite image distance into account can be derived [ 1 ] from 


Eq. 2: 


M 


- microscope 


microscope 

2NA^ ‘ 


(3) 


The majority of microscope objectives has an f-number be¬ 
tween 15 and 40. In our experiments, we use a lOx objec¬ 
tive with an f-number of F = 20. 



Figure 2. Properties of an optical system. An optical system is 
defined by its focal length /, its principal planes H and and 
the diameter D of its entrance pupil. Two conjugate planes define 
a unique magnification M, the ratio of the image size y' over the 
object size y. All optical equations can be found in [1]. 


3.2. Lytro Features 

The Lytro camera is made of an optical system that is 
forming an image in the plane of a micro-lens array that 
is, in turn, redirecting the light rays to a sensor. It has 
a 3280 X 3280 pixel CMOS sensor with 12-bit A/D and 


1.4/im X lA(im pixels [6]. Each micro-lens has a diame¬ 
ter of lAjim which is equivalent to 10 pixels. The micro¬ 
lenses are packed on a hexagonal lattice (see Fig. 3 (left)). 
The effective spatial resolution is therefore 328 x 328 pixels 
whereas the angular sampling rate is 10 x 10 values. The 
Lytro’s main objective lens has a fixed f-number of F = 2 
and features an 8 x optical zoom. We explore its potential 
as an imaging parameter in Sect. 5. Another important fea¬ 
ture of the objective lens is that it can focus from 0mm to 
infinity. 



Figure 3. Left: Image of the micro-lens array taken with a micro¬ 
scope with top illumination. We can see the hexagonal structure of 
the micro-lens array. The bright dot in the middle of each hexagon 
is an image of the light source reflected by the surface of the micro¬ 
lens. Right: The Lytro camera without its optics. 

3.3. F-number Mismatch 

A prerequisite for non-vignetted imaging (see also 
Sect. 5) is that the f-number of the micro-lens array and that 
of the microscope objective match. This is the solution em¬ 
ployed in conventional light-field microscopy [9, 18, 10]. A 
custom F = 20 micro-lens array (MLA) is typically em¬ 
ployed as it is compatible with a large number of existing 
microscope objectives. However, this F = 20 MLA is not 
readily available and has to be custom manufactured. 

Since the Lytro is designed for macroscopic imaging, its 
f-number (F = 2) is not adapted to the microscopic situ¬ 
ation. If we were to use the Lytro micro-lens array as is, 
the angular sampling in one direction would be divided by 
the ratio of the f-number of the two systems and only one 
pixel would be lit under each micro-lens (instead of approx¬ 
imately one hundred) (see Fig. 5 (bottom)). This would re¬ 
move any interest for light-field purposes since only a sin¬ 
gle view would be recorded and 99% of the sensor would 
remain unused, see Sect. 5 for examples. 

Therefore, in order to successfully use the Lytro camera 
for microscopic imaging, the f-numbers of the two optical 
systems need to be adapted. There is, however, a trade¬ 
off. The optical invariant, a fundamental law of optics (see 
e.g. [1]), states that for two conjugate planes, the product 
between the sine of the angle at which light rays reach a 
conjugate plane {a and a') and the size of the object in this 


















plane {y and y') is equal at both planes. 

y n sin{a) = y' n' sin{a'), (4) 

n and n' are the optical index of the media on both sides of 
the optical system. For air, the index is equal to 1. 

We therefore opt for an optical demagnification scheme, 
decreasing y', to increase the angular size of the cone of 
light rays a' that is incident on the Lytro’s light-field sensor. 
Theoretically, we need to divide the microscope objective’s 
f-number by 10 to reach the same f-number as the Lytro 
camera. An immediate consequence from Eq. 3 is that the 
combination of all optical elements must therefore have a 
magnification divided by 10, i.e. we are aiming to convert 
the system to unit-magnification. Due to the small size of 
the Lytro’s micro-lenses, the optical resolution of the sys¬ 
tem is still satisfactory, even at this low magnification (see 
Sect. 7). 

The magnification of the combined system Mfinal can 
be written as the product of the magnifications of each indi¬ 
vidual system: 

^final ^microscope^lensi"- ^lensN ^Lytro: ( 5 ) 

where Miensi, i=i..N indicates the magnification of N 
to-be-designed intermediate lens systems. The microscope 
objective has a fixed magnification of Mmicros cope = 10 , 
whereas the lowest magnification setting of the Lytro has a 
value of MLytro = 0.5. The resulting Mfinal = 5 with¬ 
out additional optical components (N = 0) is too large to 
prevent angular information loss. 

We explore two different options (see Fig. 4) to imple¬ 
ment the adapted system. The first option (see Sect. 5) is 
to demagnify the image of the microscope with additional 
lenses (N = 2). This solution lets us use the microscope 
and the Lytro camera unmodified. The second option is 
to remove the microscope, replacing it by an SLR lens in 
macro-imaging mode. Here, we compare a setup with and 
without the Lytro optics (see Sect. 6). 

Lytro Microscope 
(Section 5) 


-^ 



Macro Lytro 
(Section 6) 


Figure 4. The diagram shows our two proposed solutions to 
achieve low magnification. The first option (first row) keeps the 
camera and the microscope intact. The second option (second row) 
replaces the microscope with an SLR lens. 


4. Sensor Coverage 

Vignetting: When using two or more optical systems in 
conjunction, some light rays are lost because the pupils of 
the different systems do not match each other. This effect 
is called vignetting. Generally, there are two types of vi¬ 
gnetting: spatial and angular vignetting. 

Spatial vignetting directly translates into a loss of field 
of view, which may reduce the image size at the sensor (see 
Fig. 5 (top)). Angular vignetting (see Fig. 5 (bottom)) oc¬ 
curs when the cone of rays permitted through one of the 
systems is smaller than for the other system, e.g. due to a 
stop positioned inside the system. Angular vignetting is not 
an issue in a standard camera : it only affects the exposure 
and is directly linked to the depth of field of the camera. In 
a light-field camera, however, it is crucial to minimize an¬ 
gular vignetting in order to prevent the loss of directional 
light-field information. 



Figure 5. Top: Spatial vignetting occurs when light rays from the 
object (in red) do not pass through the second lens. For non- 
vignetted imaging, light rays (in blue) converging to a point at the 
sensor edge should include all the red light rays emerging from 
the object. Bottom: Representation of the sampling of a cone of 
light rays by a F = 2 micro-lens. Green rays symbolize the an¬ 
gular cone that can be acquired by the micro-lens, while blue rays 
emerge from an F = 20 optical system. As can be seen from the 
figure, angular vignetting prevents an effective sensor utilization. 

We propose to measure the vignetting in terms of its 
adaptation to the recording light-field sensor. An ideal opti¬ 
cal system that is adapted to a particular sensor would fully 
cover all its sensor elements. Since the raw pixel resolution 
is divided into spatial and angular parts, the sensor coverage 
Csensor Can be approximately expressed as 

^sensor [^] — Lspatiali^^] ^ Langular[^^]: ( 6 ) 

where Cspatiai is the spatial coverage of the sensor in per¬ 
cent, and Canguiar IS the angular coverage of one micro-lens 
sub-image, also in percent. In our experimental validation, 
we measure the spatial coverage in the center view and the 




































angular coverage in the center lenslet. This choice is moti¬ 
vated by the simpler estimation of the relevant coverage ar¬ 
eas as compared to using the side views/edges of the field. 
The measure can be considered to be an approximation of 
the upper bound of the system space-bandwidth product, i.e. 
the optical information capacity of the system [13]. 

4.1. Unmodified Use of the Lytro’s Main Optics 

The main optics of the Lytro camera, i.e. the optics 
without the micro-lens array, is designed to avoid angular 
vignetting when imaging onto the micro-lens array, i.e. the 
micro-lens array and the main optics have been designed 
with the same f-number of F = 2. We have observed 
that the main optical system can be used in two different 
ways with a microscope. These two imaging regimes can 
be used differently in designing an optical matching system. 

Regular regime: The Lytro camera can image a plane 
as close as the first surface of its optics for a zoom level of 
1X. This minimal focus distance increases with the zoom 
level. In order to use the Lytro with the microscope, it has 
to be positioned such that the near focus of the camera 
is placed at the image plane of the microscope objective. 
Since the image size y' of the microscope objective is 
rather large (typically around 50mm x 50mm) whereas the 
Lytro’s entrance pupil is only 20mm in diameter, spatial 
vignetting incurs a loss of sensor coverage of up to 94% as 
shown in Figure 7 (left). The angular vignetting is stronger 
with only 16% angular coverage. 

Inverse regime: We discovered that, in a specific con¬ 
figuration where the camera is set to focus to the closest 
possible plane for a zoom level of 1 x, the camera can enter 
into a virtual object regime. The camera is then able to im¬ 
age an object plane that is located behind its first lens, i.e. in 
the direction of the sensor (see Fig. 6). This configuration 
enables the positioning of the camera close to the micro¬ 
scope objective and therefore reduces the spatial vignetting 
since a larger number of rays can be captured by the lens 
surface (see Fig. 7 (right). This mode of operation inverts 
the image. 

For both imaging regimes described above, the magnifi¬ 
cation MLytro (~ 0-5) is not low enough for achieving a 
good angular coverage. While the spatial vignetting prob¬ 
lem can be successfully addressed with the inverse regime, 
the angular vignetting can only be dealt with by using a low 
magnification optical system. We therefore investigate the 
different options. 

5. The Lytro Microscope 

Our first option to achieve the matching of the f-numbers 
discussed in Section 3.3, consists in designing an optical de¬ 
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Figure 6. Top: Setup for the inverse regime configuration. Bottom: 
Distance of Lytro focus plane to its front lens with the variation of 
zoom. An abrupt change of position of the zoom lens group occurs 
for a zoom level of 3.7x, enabling to switch between the regular 
and inverse regime. 
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Figure 7. Direct imaging through a 10 x microscope objective with 
an unmodified Lytro. The object is a blue LCD display. It con¬ 
sists of a square black grid that is separating the different pixels 
of 0.5mm x 0.5mm size. We hypothesize the white dots to be 
bubbles inside the liquid crystal. The large images show the equiv¬ 
alent camera image computed from the light-field, while the small 
images on the bottom right show a close-up on the micro-lens im¬ 
ages of the raw sensor data for different zoom-levels. Spatial and 
angular vignetting are easily observed in the equivalent camera 
and micro-lens images, respectively. The zoom level setting is 1 x 
(regular regime) on the left and 5 x (inverse regime) on the right. 
The spatial vignetting is strong in the regular regime (6% of spa¬ 
tial coverage), while it is greatly improved in the inverse regime 
(100% of spatial coverage). Angular coverage is similar in both 
cases {Cangular ~ 25%). 

magnification system (placed between the microscope ob¬ 
jective and the Lytro) that increases the angular extent of 
the light (see Eq. 4). This solution keeps desirable proper¬ 
ties like the large numerical aperture of the microscope and 
its fixed working distance, while at the same time, the Lytro 
camera can remain unmodified. The major task is to find a 
good trade-off between the vignetting and the magnification 
of the resulting light-field microscope. 




















Our best solution along this direction employs two 
lenses. This configuration serves two goals: 1) to decrease 
the magnification successively, simplifying the task of each 
individual lens, and 2) to move the image behind the Lytro 
camera so that it can be used in the inverse regime which 
offers a better spatial coverage Cspatiai • The ray-diagram in 
Fig. 8 illustrates the two-lens setting: an intermediate image 
that is slightly demagnified is created in front of the micro¬ 
scope. Then, the second lens creates a further demagnified 
image behind the Lytro camera. The Lytro camera is oper¬ 
ated in its inverse imaging regime in order to pick up this 
virtual image. We determine the positions and focal lengths 
of the additional lenses using the following equation derived 
from the thin lens formula: 


The focal length / and the position of the lens x are chosen 
according to the desired magnification M (negative because 
the image is inverted). In our implementation, the first ad¬ 
ditional lens has a focal length of 50mm and is put close 
to the microscope objective. The second additional lens has 
a focal length of 85mm and is put close to the Lytro. The 
effect of using two lenses is that the individual focal lengths 
are larger and the aberrations are reduced. 


Microscope Additional Additional Lytro 
B r Objective Lens 1 Lens 2 Optics 

^ A A A 



Figure 8. Ray-diagram of the system using two additional lenses. 
Each sub-system is indicated in a different color and has to be 
interpreted independently from the other sub-systems. Their oper¬ 
ation can be understood in sequence: the objective images object 
AB to A'B' , the first additional lens images A'B' to AiBi so that 
the second lens images AiBi to A 2 B 2 behind the Lytro camera. 
Finally, the Lytro’s main optics in the inverse regime images this 
virtual object A 2 B 2 to its sensor in the plane A"B". 

6. Macro Lytro 

Our second option is to use a single SLR camera lens in 
front of the Lytro to achieve unit magnification. (Mfi^ai = 
MsLRMLytro = !)• As for the microscope, and because 
the focal length of the SLR lens is large (50mm and 100mm 
in our experiments), we want to use the Lytro in the inverse 
regime to keep the spatial coverage as high as possible. In 
practice, the Lytro is set as close as possible to the SLR 
lens. A variation of this setup is to remove the Lytro optics, 
and only use the SLR lens so that Mfinal = Mslr = 1- 


These designs have only one or two optical components and 
relieve the hurdle of undoing the work of the microscope 
objective with many lenses. However, the SLR lens is not 
specifically designed for the magnification of close objects 
and its aperture is not meant to be maintained at a constant 
value for the micro-lens array. The relations used to estab¬ 
lish Eq. 3 are not valid in this macro-configuration. Instead, 
the relevant quantity is the working f-number F^j [1] which 
is the f-number modified by the magnification M : 

^-" = 27 ^ = ® 

where NAi is the image-side numerical aperture NAi = 
n'sina'. The minimal value of working f-number that can 
be achieved with a camera lens is close to = 2. It is 
reached for a limit f-number of F = 1 and a magnification 
of M = —1. This condition would be optimal for the Lytro 
micro-lens array. However, commercial lenses usually have 
a limit f-number between 1.4 and 3.5 increasing the working 
f-number to between 2.8 and 7.0. 

Compared to the light-field microscope, on the one hand, 
this setup is more versatile. The magnification can be set 
to the desired value by simply moving the object and the 
micro-lens array. It does not require the difficult alignment 
of several optics. On the other hand, this setup does not 
benefit from the structure of the microscope that already in¬ 
cludes lighting and moving the sample through micrometer 
stages in three dimensions. In addition, the working dis¬ 
tance is not fixed which changes the magnification as well 
as the object-side numerical aperture when moving the sam¬ 
ple. However, the strong point of this design is its acces¬ 
sibility. Building a light-field macrography setup is done 
quickly and without the need for a deep understanding of 
the operating principles of a microscope. 

7. Results 

Before showing results, we describe and compare the 
different implementations. The different sensor coverages 
as well as their spatio-angular coverage values can be found 
in Fig. 9. It is clearly visible that directly using the Lytro 
camera in its regular imaging regime is unsuitable for mi¬ 
croscopic light-field imaging. The inverse imaging regime 
improves on the spatial coverage, but the angular coverage 
is limited. The best combination is achieved with a 50mm 
SLR lens in front of the Lytro which yields the best overall 
sensor coverage Csensor- 

7.1. Resolution Test Chart 

In order to compare the resolution of the two different 
techniques, we use a 1951 USAF resolution test chart. The 
results of the experiments are summarized in Fig. 10. 

The first option (Sect. 5) ’’Lytro Microscope” was imple¬ 
mented using a Canon 50mm SLR lens and a Nikon 85mm 
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+ regular 
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XLytnoMicro 2.88x 

• SOmmSLR +MLA1.31X 
50mm SLR + MLA 2.34x 
50mm SLR + Lytro 1.26x 

5|c 100mm SLR + Lytro 1.36x 
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Figure 9. Combined results of the experiments from Sect. 5 and 
Sect. 6. 


degrades the image. The aberrations are reduced in the SLR 
+ Lytro case as compared to SLR + MLA, since the magni¬ 
fication of the SLR lens is lower in this setting. It is most 
noticable in the side views since, for these views, imaging is 
performed through the outer pupil regions of the SLR lens. 
We suspect that the chromatic aberration is introduced by 
the SLR lens because it is not intended for macro-imaging. 
Using a dedicated macro-lens instead would likely remove 
this effect. 


Setup 

Magnification 

Resolution (Ip/mm) 

Microlens 
footprint (pm) 

Measured 

Theoretical 

Lytro Microscope 

2.88 

80 

103 

4.87 

50mm SLR + MLA 

1.31 

32 

47 

10.7 

50mm SLR + MLA 

2.34 

51 

84 

5.99 

50mm SLR + Lytro 

1.26 

32 

45 

11.1 

100mm SLR + Lytro 

1.36 

36 

49 

10.3 


Figure 10. This table summarizes the results from the experiments 
of Sect. 5 and Sect. 6. Theoretical resolution is derived from the 
micro-lens pitch. The micro-lens footprint is the size of a micro¬ 
lens in object space. 



Figure 12. Setup from the SLR + Lytro experiment using a 100mm 
SLR lens (100mm SLR + Lytro). 


SLR lens as additional lenses. Those lenses were put on 
top of a Leitz Ergolux microscope using an objective of 
magnification 10 x with an object-side numerical aperture 
NAo = 0.2. This microscope has a lens tube with a mag¬ 
nification of 0.8X so the f-number F = 20. The images 
have been taken with a magnification of 2.88, i.e. a micro¬ 
lens covers 4.87/im in object space (see Fig. 11 (top)). The 
spatial coverage is above 99% but due to the large magnifi¬ 
cation the angular coverage is low (between 9% and 25%). 
The resolution is between 80 and 90 line pairs per mm. 

The resolution indicated above is computed for the cen¬ 
ter view. It decreases with further distance from the center. 
A loss of image quality due to aberrations can be observed. 
They are introduced because the observed area is larger 
than usual for the microscope objective. Microscope ob¬ 
jectives are typically designed so that only a reduced inner 
portion of the full field is very well corrected. In addition, 
our matching lenses introduce further aberrations. Since the 
angular vignetting is strong, the contrast of viewpoints far 
from the center is low. It should be noted that even view¬ 
points inside the vignetted area can be computed, albeit at a 
poor signal-to-noise ratio (see Fig. 11). 

The second option (Sect. 6) was implemented in three 
ways: two times with the Lytro placed behind two different 
lenses, a 50mm and a 100mm Canon SLR lens (referred as 
SLR -I- Lytro), and with the Lytro micro-lens array without 
the Lytro main optics, see Fig. 3 (right), placed behind the 
50mm lens (referred as SLR MLA) (see Fig. 12). Magni¬ 
fications from 1.26 to 2.34 were achieved. The spatial cov¬ 
erage is always 100% and angular coverage is good (up to 
70%). In this case, chromatic aberrations are present which 


7.2. Microscopic Sample 

The most direct application of the light-field microscope 
is the study of microscopic samples. The low magnification 
and the large field of view allow us to see in detail an object 
area that is between 1.5mm x 1.5mm and 3.5mm x 3.5mm 
with a magnification of 2.88x and 1.3x respectively. Cell 
tissues or rough surfaces of different materials have a struc¬ 
ture close to the millimeter so high magnification is not al¬ 
ways necessary to analyze them. 

We illustrate this technique in Fig. 1 (right). Several im¬ 
ages of microscopic specimen were taken with the same set¬ 
tings as in the previous section. The magnification is 2.88 x 
and we can clearly see the structure of different kind of sur¬ 
faces that are invisible to the naked eye. 

The light-field data allows for the reconstruction of the 
depth of the sample when the number of views is suffi¬ 
cient. We took a picture of a ground truth aluminium stair¬ 
case with stairs of 1.00mm width and 0.50mm height with 
an accuracy of ±5/im with the SOmmSLR -i- Lytro setup. 
We obtained the depth map in Fig. 13 using a modified 
variational multi-scale optical flow algorithm for light-field 
depth estimation [15]. Although only a small slice of the 
staircase is in focus (the depth of field is 1mm), the depth 
can be computed outside of this area. Essentially, out-of- 
focus regions are naturally considered as a different scale 
by the algorithm, so, the estimation of the depth of the clos¬ 
est and furthest steps is correct. This behavior nicely inter¬ 
acts with the scene properties since the parallax is larger in 
out-of-focus regions. The optical system can be seen as sup¬ 
porting the part of the algorithm that is handling large dis- 




























Figure 11. Set of different viewpoints of the resolution target with the center view in the middle (the red dot in the top left inset indicates 
the position of the view). Top: Images taken with the ’’Lytro Microscope” (Sect. 5). The magnification is 2.88. Bottom: Images taken with 
the ”50mmSLR + Lytro” (Sect. 6). A red-green color shift due to strong chromatic aberrations can be observed in the side views. Note that 
the top row has a higher resolution: it shows the pattern that is visible in the center of the bottom row (level 4 and 5). The contrast of the 
images of the same row have been set to a similar level for comparison. 


placements. Since the detailed properties of the Lytro main 
optics are unknown, it is necessary to adjust the scale of the 
computed depth. We use the aluminium staircase as refer¬ 
ence. We find the affine transformation between the depth 
map and the staircase model by fitting planes to match the 
stairs. After the transformation, we measure an RMS error 
of 75/im for vertical planes and 17/im for horizontal planes. 
The difference is due to the different slopes of the horizontal 
and vertical steps with relation to the camera view direction. 
The magnification is equal to 1.32. We also applied this 
depth reconstruction to a daisy head (see Fig. 13). In prac¬ 
tice, the depth inside a cube of about 3.5 x 3.5 x S.bmm^ 
can be estimated, which is rather large for a microscopic 
setting. 

8. Discussion and Conclusion 

We have developed and tested several adaptations of 
the Lytro consumer light-field camera to enable an entry- 
level experimentation with light-field microscopy. While 
the fixed f-number of the Lytro’s micro-lens array prevents 
its direct use with a standard microscope (regular regime), 
it is possible to trade the overall system magnification for 
light-field features and to avoid spatial vignetting with the 
inverse imaging regime. Lytro microscopy is therefore an 
option for low-magnification work as is common in indus¬ 
trial settings, or for investigations into the meso- and large- 
scale micro-structure of materials. Even though an optical 
magnification between 1 and 3, as achieved in this work, ap¬ 
pears to be low, the small size of the micro-lenses still yields 
a decent optical resolution of up to 6.25/im in object space 
which already shows interesting optical structures that are 
imperceivable by the naked eye. 



Figure 13. Sub-view (top left) and computed depth map (top mid¬ 
dle) of a daisy flower. Sub-view (bottom left) and computed depth 
map (bottom middle) of the aluminium staircase. The bottom right 
picture is a 3D visualization of the depth map after the calibrating 
and the top right picture is a projection of a region in the middle of 
the calibrated depth map onto the xz plane showing the plane fits 
in red. 

For the future, we would like to investigate image-based 
denoising schemes for the vignetted side-views, as well as 
algorithmic developments for structure recovery. In terms 
of applications, the imaging of micro-and meso-BRDFs and 
their relation to macroscopic BRDF models appears to be an 
interesting development. 
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